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DETECTION  IN  NON-GAUSS IAN  BACKGROUND 
Philip  Rudnick 


ABSTRACT 


The  problem  of  detecting  a small  signal  in  a non-Gauss ian 
background  is  discussed  analytically  for  a very  limited  set  of 
conditions.  It  is  supposed  that  the  background  contains  bursts 
of  noise  whose  complete  rejection  might  aid  detection.  For  sev- 
eral specific  detection  procedures,  fcmulas  are  derived  by  which 
empirically  known  distribution  functions  of  the  background  may 
be  used  to  predict  a change  in  detection  threshold  resulting 
from  burst  rejection.  Particular  attention  is  given  to  the  useof 
the  first-order  distribution  function,  even  though  it  alone  does 
not  completely  determine  the  results.  A suggestion  by  Birdsall 
connecting  this  study  with  the  use  of  the  likelihood  ratio  is 
also  discussed. 
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DETECTION  IN  NON-GAUSS IAN  BACKGROUND 
Philip  Rudnick 


INTRODUCTION 


Current  theories  for  the  detection  of  a signal  in  noise 
have  recently  been  summarized  by  Peterson,  Birdsall  and  Fox .2/ 
These  theories  employ  a very  elaborate  specification  of  the  noise 
statistics,  namely  a probability-density  function  in  n dimensims, 
where  n is  the  number,  ordinarily  quite  large , of  parameters  re- 
quired to  specify  the  receiver  input  in  complete  detail  through- 
out the  allowed  integration  time.  The  case  of  Gaussian  statis- 
tics leads  to  distribution  functions  which  can  be  expressed  in 
terms  of  such  accessible  quantities  as  power  and cro  ss  correlation 
coefficient.  Consequently  the  general  theory  can  be  carried 
through  to  practical  conclusions  in  the  Gaussian  case,  and  one 
has  assurance  that  the  indicated  methods  are  the  best,  in  some 
defined  sense. 


If  tbe  noise  background  has  non-Gaussian  statistics,  the 
practical  situation  is  very  different.  The  designer  of  a detec- 
tion apparatus  is  not  likely  to  have  full  knowledge  even  of  the 
first  and  second  order  statistics  of  the  backgroimd,  and  will 
quite  certainly  have  no  information  of  n-th  order  complexity. 
He  will,  in  addition,  wonder  tdiether  these  statistics,  when  ob- 
tained, will  prove  to  be  sufficiently  stable  and  otherwise  prac- 
tically adaptable  for  use  in  apparatus  design.  In  this  situation 
realizable  first  steps  in  the  adaption  of  detection  processes 
to  non-Gaussian  noise  are  needed,  even  though  they  be  neither 
general  nor  optimum. 

Ihis  study  deals  mainly  with  a quite  limited  and  specific 
question,  that  of  detection  in  the  presence  of  a background  which 
has  a somewhat  impulsive  or  burst-like  character,  and  of  rejecting 
these  impulses  as  a means  of  improving  detection.  Particular 
attention  is  given  the  first  order  distribution  (of  instantaneous 
values  of  background)  as  a guide  to  design  of  detection  procedure. 
Of  course,  higher  order  statistics  are  also  relevant  and  if  these 
are  not  imown,  the  mathematical  foundation  is  necessarily  in- 
complete . 


The  calculations  that  follow  are  concerned  with  back- 
ground noise  whose  distribution  functions  are  unspecified  and 
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largely  arbitrary.  The  results  are  expected  to  be  of  value  pri- 
marily vdien  they  are  used  in  conjunction  with  empirically  observed 
distribution  functions  for  the  background  actually  present  in 
some  detection  problem.  In  this  way,  one  may  predict  detection 
thresholds  for  small  signals,  from  experimental  observations  on 
the  background  statistics  alone,  and  without  any  actual  detec- 
tion experiments. 


NOTATION  AND  FORMULATION  OF  PROBLEM 

The  diagram  in  Figure  1 indicates  the  principal  class  of 
detection  processes  to  be  discussed  (though  one  will  also  be  in- 
cluded to  which  Figure  1 is  not  appropriate). 


x(t)^ 

generalized 

y(t) 

averager 

z(t) 

rectifier 

Figure  1.  General  block  diagram  of  detector. 


x(t)  represents  the  detector  input.  It  will  be,  alter- 
natively, either  one  of  two  stationary  time  series, 
representing  background  noise  alone,  or  background 
plus  the  small  added  signal  which  is  to  be  detect- 
ed. In  either  case,  the  time  average  x(t)  is  zero. 
No  frequency-selection  procedures  are  considered 
here,  aid  it  will  be  supposed  that  any  appropriate 
filtering  has  already  been  done.  x(t)  will  be  con- 
sidered either  a continuous  function  of  time,  or 
a series  of  discrete  samples  taken  at  equally 
spaced  instants. 

Xe(t)  is  the  envelope  of  x(t),  in  the  sense  used  in  Ap- 
pendix I. 
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y(t)  is  the  output  of  a generalized  rectifier.  The 
instantaneous  value  of  7(t)  is  either  a function 
of  the  absolute  value  of  the  instantaneous  input 
|x(t)|,  or  of  Xe(t),  and  will  also  be  written  7[x] . 
Thus  the  rectifier  ma7  include  ”integration”  over 
one*'period"  of  a narrow-band  input,  but  not  longer. 

z(t)  is  the  result  of  ruining  linear  averaging  of  7(t). 
The  averager  ma7  be  a low  pass  filter. 

T is  the  effective  averaging  time  of  the  averager 
and  must  be  chosen  consistent  with  some  response- 
time stipulation  in  the  basic  detection  problem. 
Since  small  signals  are  contemplated,  T will  al- 
wa7s  be  long  con^ared  to  the  correlation  time  of 
x(t),  and  to  the  san^ling  interval,  if  x(t)  is 
discrete.  T is  also  the  correlation  time  of  the 
output  time  series  z(t).  Loosel7  one  ma7  sa7  that 
z(t)  presents  one  new  independent  value  (on  which 
a judgment  of  ’’signal  present”  or  ’’signal  absent” 
ma7  be  based)  each  T seconds,  and  that  each  such 
value  is  derived  from  the  input  x(t)  during  a like 
interval  T. 

n is,  if  x(t)  or  Xe(t)  is  sampled,  the  number  of 
such  samples  occurring  in  time  T. 

to  ■ T/n,  is  the  interval  between  samples. 

f(u)  is  the  first  order  probabilit7  densit7  for  instan- 
taneous values  of  |x(t)|  for  background  onl7;  f(u)du 
is  the  fraction  of  time  during  vhich  u<|x(t)|<u  du. 
The  derivative  of  f(u)  is  assumed  to  vanish  as 
u -►0, 

f(u)  g(u)  is  the  corresponding  probabilit7  densit7  when 
|x(t)|  also  contains  the  signal. 

F(u)  and 

F(u)  G(u)  are  corresponding  probabilit7  densities  for  the 
envelope  x^Ct),  without  and  with  signal.  F(u)  is 
assumed  to  vanish  as  u-*-0. 

f2(u,v,  r)  ■ fo(u,v,-r)  is  a STnmetrized  second  order  proba- 
bilit7  densit7  for  background  onl7;  f2(u,v,r)du  dv 
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is  the  joint  probability  that  u<|x(t)|<u  ♦ du  and 
V < |x(t  + r )1  <u  du,  or  that  u <|x(t  ♦r)|<u 
du  and  u<|x(t)|<u  du;  as  an  alternative  £2  has 
the  corresponding  meaning  for  Xg. 

E(w)  and  V(w)  are,  respectively,  expected  value  or  mean,  and 
variance  of  any  quantity  w depending  on  the  input 
or  its  envelope,  in  the  absence  of  signal,  w may 
be  a function  of  the  instantaneous  value  of  |x(t)| 
or  Xg(t),  or  it  may  be  a quantity  determined  by 
thevdiole  set  of  input  values  extending  through  an 
integration  interval  T. 

Eg(w)  and 

Vg(w)  are,  respectively,  mean  and  variance  as  above,  ex- 
cept that  the  signal  is  present. 

2 is  the  variance  or  ’’power”  of  the  signal  idiich  may 
be  contained  in  x(t).  This  is  the  only  property 
of  the  signal  >diich  needs  enter  our  main  discus- 
sion. 

(1) 

is  an  output  signal-to-noise  ratio  vdiich  will  be 
used  here  as  a criterion  of  detection  capability. 
The  signal  is  assumed  small  enough  to  make  a dis- 
tinction between  Vs(z)  and  V(z)  unnecessary  in 
this  expression. 

N^,  Ng,  etc.  are  special  values  of  the  numerator  in  Eq.  (1)  re- 
ferring to  specific  detection  processes  A,  B,  etc,, 
to  be  defined  later, 

Va(0,  VA(y), 

Vb(*)>  *re  special  values  of  V(z)  or  V(y)  also  referring 

to  specific  detection  processes  A,  B,  etc. 


Further  notation  is  defined  in  the  following  section. 
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REJECTION  PROCESSES,  DESCRIPTION 

If  a background  noise  contains  impulses  or  bursts  of  suf- 
ficient prominence,  it  can  be  conjectured  that  such  bursts  may 
contribute  disproportionately  to  the  fluctuation  Vg(z)  and  their 
elimination  may  have  a benefit  outweighing  the  loss  of  signal 
^chmust  also  occur.  Hence,  the  next  two  sections  are  devoted 
to  calculating  the  reduction  in  signal  and  reduction  in  noise, 
respectively,  idiich  result  when  noise  bursts  are  rejected.  In 
this  section  several  specific  processes  are  described,  all  di- 
rected generally  toward  burst  elimination.  For  this  purpose,  we 
define  further  notation,  and  assign  more  specific  properties  to 
the  detection  process. 

a is  a rejection  limit,  defined  by  the  condition 
y(t)  =0  whenever  |x(t)|  >a,  or  alternatively 
Xe(t) > a. 

^ a ^ a 

q “ / f(x)  dx  or  / F(x)  dx,  is  the  expected 
o o 

fraction  of  the  input  x(t)  or  Xg(t)  not  rejected, 
when  no  signal  is  present. 

p ■ 1-q,  is  the  expected  fraction  of  input  rejected. 

w * (l/q)  J wf(x)  dx  or  the  corresponding  integral 

with  F(x)  replacing  f(x).  It  is  the  special  case 
of  E(w)  irfien  averaging  is  over  nonrejected  input 
values  only. 

pxi'r) t p air) , etc.  are  autocorrelation  coefficients,  for  noise 
only,  of  y(t)  in  processes  A,  B,  etc.  They  are 
normalized  to  P (0)  * 1.  given  by  Eq.  (9). 

For  p B,  see  Appendix  II. 

^A>  correlation  times,  defined  as  in  Eq.  (8)  or 

(11),  for  processes  A,  B,  etc. 

q’  is  an  actual  fraction  not  rejected  during  an  inter- 
val T,  for  a fixed  rejection  limit  a.  It  is  a 
random  variable. 

a*  is  a variable  rejection  limit,  fixed  during  any  one 
sample  (of  £q>pn)ximate  duration  T)  but  varied  from 
saiiq>le  to  sanqsle  in  such  a way  as  to  reject  the 
same  fraction  p from  each  sanqile. 
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We  now  specify  three  distinct  detection  processes  A,  B, 
and  C which  may  be  considered  at  least  loosely  as  specializations 
of  the  scheme  outlined  in  Figure  1,  and  one  additional  process, 
D,  of  somewhat  different  character.  Each  process  is  understood 
to  include  operation  on  x(t)  or  x^Ct),  either  of  which  may  be 
continuous  or  discrete. 


Process  A 

All  values  of  y(t),  including  the  zero  values  resulting 
from  rejection,  enter  the  averaging  process.  The  duration  or 
number  of  such  zero  values  contained  in  one  integration  interval 
T and  affecting  one  instantaneous  value  z(t)  is  of  course  subject 
to  sampling  fluctuations. 

Process  B 

The  zero  values cf  y(t)  resulting  from  rejections  do  not 
enter  the  averaging  process,  and  z(t)is  an  average  cf  non-re jected 
values  of  y(t).  These  values  are,  of  course,  not  uniformly  dis™ 
tributed  in  real  time  and  might  be  dealt  with  ins trumen tally  by 
an  averager  capable  of  intermittent  operation  in  real  time.  We 
will,  however,  presume  the  non-rejected  y(t)  values  to  be  stored 
and  then  re-presented  uniformly  on  a new  time  scale  in  which  the 
gaps  arising  from  rejection  have  been  closed,  as  illustrated  in 
Figure  2.  The  labeling  of  the  new  time  scale  will  differ  from 
realtime  only  by  shifts  of  origin  as  one  passes  from  one  retained 
segment  to  another . On  this  scale  the  averaging  will  be  conducted 
with  an  integration  time  q'f,  corresponding  to  qT/q*  for  the  orig- 
inal data.  The  difference  between  this  latter  time  and  Tis  con- 
sidered negligible.  The  maximum  amplitude  transmission  factor 
of  the  integrating  filter  is  for  convenience  taken  to  be  q. 

Process  C 

The  rejection  limit  is  the  variable  a’,  defined  above, 
which  results  in  a fixed  fraction  p of  zero  values  of  y(t).  These 
are  included  in  a simple  average,  as  in  process  A.  (This  same 
procedure  might,  however,  equally  well  be  considered  a variant 
of  process  B.) 

Process  D 

In  this  case  the  final  output  of  the  detection  process 
is  taken  to  be  the  rejected  fraction  (1  - q*)  with  q®  as  defined 
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Figure  2.  Process  B.  I/iusfrathn  z>r  reject iiw  {a) 
j/jJ  siosure  of  resuiting  gaps  (d). 
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above.  As  will  be  shown  later,  an  equivalent  choice  is  the  var- 
iable limit  a’  corresponding  to  fixed  rejection  fraction,  also 
defined  above. 


REJECTION  PROCESSES,  OUTPUT  SIGNAL 

The  numerator  of  the  right  member  of  Eq.  (1),  is  an 
expected  increment  in  output  resulting  from  the  presence  of  a 
signal.  This  will  now  be  evaluated  for  the  specific  processes 
just  described.  The  results  prove  to  bo  expressible  in  terms  of 
the  first  order  distribution  of  the  background  and  the  power  of 
the  signal. 


Process  A 


If  y depends  on  x rather  than  its  envelope,  we  write 


E(z) 

and 

By  subtraction 

Na  - Es(2)  - E(z) 


E(y) 

/ ^ y[x]  f(x)  dx 

y[x]  [f(x)  + g(x)]  dx. 

/ yW  6(x)  dx. 

•'0 


(2) 


Process  B 

The  situation  now  differs  from  that  above  in  that  the 
distribution  functions  are  now  normalized  to  unit  total  probab- 
ility over  the  interval  0<  x<a,  and  the  filter  transmission 
factor  q is  inserted.  Hence 
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J y f(x)  dx 
o 

E(x)  - q 

f f(x)  dx 
^ o 

/o  y[f(x)  ♦ g(x)J  dx 
Eg(z)  - q 2 * 

/o  ^ 

Treating  g(x)  as  a small  quantity,  we  obtain 

Nr  ■ Es(a)  - E(*)  “ [yTx]  - y]g(x)  dx.  (3) 


Process  C 

Here  one  may  recognize  for  each  value  of  a’  a mean  value 
of  z which  depends  on  a*.  The  final  E(z)  is,  strictly,  a mean 
of  these  means,  but  to  a satisfactory  approximation  when  the 
relative  fluctuations  in  a*  are  small,  we  will  calculate  E(z) 
as  a single  mean  associated  with  the  limit  a,  just  as  in  A. 

E(z)  - y(x)  f(x)  dx. 

A similar  approximation  will  be  made  in  the  presence  of  signal, 
but  with  a higher  limit. 

with  8 determined  by 
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or 

f(a)  S " - /^  g(x)  dx. 


(4) 


Hence 


Nc  - Es(z)  - E(z) 


For  later  convenience,  Eq.  (2),  (3),  and  (5)  are  integrated  by 
parts  to  gives 


/o 


dy  W 

dx 


dx 


(5M 


- Nc  - y[aj/o  [jg(x)]  dx  (2*) 

Ng  = Nq  - [y[a]  - yj/jj  [-g(x)j  dx.  (3») 

These  results  apply  for  either  continuous  or  discrete  input. 
When  y depends  on  Xg,  g(x)  is  replaced  by  G(x). 


Process  D 

If  (1  - q’)  is  the  chosen  detector  output,  we  have 
E(1  - q’)  » 1 - q 


and 

Es(l  - q’)  “ 1 - /q  g(*)]  dx. 
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Hence 

Nd  - ^ [-gU)]  dx.  (6) 

If  a*  is  the  chosen  output,  we  have  approximately 
Es(a»)  - E(a»)  - (a  +8)  - a - 8 

= [Vf‘(a)]/^*[-g(x)]  dx 

where  8 is  the  quantity  in  Eqo  (4).  This,  of  course,  differs 
from  Eq.  (6)  only  by  a constant  factor. 

Next,  we  draw  on  results  developed  in  Appendix  I,  which 
express  g(x)  and  G(x)  in  terms  of  f(x)  or  F(x),  and  the  signal 
power  We  may  note  the  typical  behavior  for  the  increment 
g(x)  indicated  in  Figure  3.  The  net  area  under  the  curve  of 
g(x)  against  x is  zero,  with  the  negative  portion  lying  to  the 
left  of  the  positive.  Hence  the  integral 


I 

i 


/„*[-«<*>]  ^ 

which  appears  in  Eqs.  (2»),  (3*),  (5*)  and  (6)  is  normally  a 
positive  quantity.  This  leads  to  the  inequalities 


Na<  Nb<  Nc 


(7) 


These  relations  are  readily  understood  intuitively.  Eq. 
(6)  shows  that  the  presence  of  signal  causes  a systematic  in- 
crease in  the  rejected  fraction  of  input.  Ng  measures  an  increase 
in  the  mean  non-re jected  y,  due  to  signal.  is  less  because 
it  is  also  affected  by  an  increased  proportion  of  keros  caused 
by  the  presence  of  signal.  Nq  is  greater  because  it  is  also  af- 
fected by  the  increased  rejection  limit. 

Equations  (2*),  (3’),  (5’),  and  (6)  have  been  combined 
with  (37)  or  (38),  and  y has  been  specialized  to  either  x^  or 
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(x|,  appropriate  to  a square  law  or  linear  detector.  The  re- 
sulting specific  expressions  for  the  numerator  in  Eq.  (1)  are 
suimarized  in  Table  1. 


REJECTION  PROCESSES,  OUTPUT  NOISE 

The  denominator  in  Eq.  (1)  represents  the  root-mean- 
square  fluctuation  in  the  output  z in  the  absence  of  signal. 
Unlike  the  output  signal  evaluated  in  the  preceding  section, 
this  output  noise  does  not  in  general  depend  solely  on  the  first 
order  distribution  of  y,  but  on  higher  statistics  as  well.  In 
the  special  case  where  the  input  consists  of  discrete  and  inde- 
pendent values  of  the  variable  x,  the  first  order  distribution 
does  determine  all  the  statistics.  For  this  reason,  this  spec- 
ial case  will  be  included  in  the  following,  even  though  in  most 
real  detection  problems,  spacing  of  input  values  would  be  so 
chosen  as  to  yield  at  least  moderate  serial  correlations . 

Process  A 

For  a continuous  input,  the  mean  squared  output  fluctu- 
ation V(z)  may  be  calculated  as  the  product  of  two  factors,  one 
the  effective  band  width  of  the  averaging  filter,  (1/T),  and  the 
other  the  power  spectrum  density  of  the  filter  input  y(t)  at 
zero  frequency.  The  latter  is  obtained  through  the  Fourier  trans- 
form of  the  autocorrelation  of  y.  Thus 

v*(.)  - 7 V*(,)  /^*(r)  dr  - a V4(,)  (8) 

with 


" /o  /o  yM  y[y]  f2(u,v,r)  du  dv  - q^  y^  (9) 


and 


Vy)  ■ q - (q  y)^  - q (y^  - y^)  ♦ p y^ 


(10) 


When  the  input  is  discrete,  the  integration  in  Eq.  (8)  is  re- 
placed by  a summation;  we  then  have 


13 


SIO  Reference  58-34 


Table  1= 


VALUES  OF  OUTPUT  DUE  TO  SIGNAL 
FOR  PROCESSES  A,  B,  AND  D 
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Table  I.  (Continued) 
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'"a  " .1  * 2 


CO 


(11) 


Combining  Eqs„  (8)  and  (10)  gives 


Va(z) 


q[(y^  ~ J^) 


p f 


^"1 


(12) 


In  the  special  case  of  discrete,  indepenaent  input  values, 
“ tg  and  we  have 


Va(z) 


p y 


(13) 


In  order  to  calculate  V.(z)  as  a function  of  the  re- 
jection limit,  knowledge  of  the  first  order  distribution  f(x)  or 
F(x)  serves  to  evaluate  all  factors  but  in  Eq  (12),  In  the 
event  that  the  first  order  distribution,  but  not  the  second,  is 
known  for  some  particular  background,  it  would  appear  to  be  of 
some  use  to  evaluate  Eq..  (12)  as  a function  of  the  rejection 
limit  a in  so  far  as  possible,  but  to  regard  the  correlation 
time  as  constant,  independent  of  a.  In  so  doing  one  is  neg- 
lecting a factor  which  may  easily  be  substantial,  but  does  not 
appear  at  all  likely  to  be  overriding.  Any  reduction  in  fluctu- 
ation which  might  be  indicated  in  this  way  would  be  an  over- 
estimate only  if  an  effect  of  rejection  were  to  increase  auto 
correlation  coefficients  and  hence 

Process  B 

Vb(z)  is  calculated  in  the  same  general  way  as  above, 
with  some  specific  changes.  The  averaging  time  of  the  filter 
is  qT  rather  than  T;  its  maximum  power  transmission  factor  is 
q2  rather  than  1.  The  autocorrelation  of  y has  been  altered 
(see  Figure  2),  so  that  a different  correlation  time  *’b  is  in- 
troduced . 
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We  now  have 

VB(y)  - - 7^ 

Vb(x)  - ^ reVgCy)  - q (y^  - (14) 

Again,  for  discrete,  independent  inputs, 

VgCx)  - J (y^  - y^)-  (15) 

In  this  extreme  case,  Eq.  (13)  is  greater  than  Eqo  (15) 
by  the  last  term  which  can  be  interpreted  as  resulting  from  the 
fluctuation  in  the  number  of  zeros  which  enter  the  process  A av- 
erage. At  the  other  extreme,  for  continuous  input  it  is  shown 
in  Appendix  II  that  the  autocorrelations  of  y (and  hence  ^4  and 
fg)  differ  in  processes  A and  B in  such  a way  that  Eqs.  (12)  and 
(14)  give  identical  fluctuation  values,  and  Va(z)  * Vb(z), 

Process  C 

Vq(z)  is  calculated  in  Appendix  III  for  the  discrete- 
independent  input  case,  with  the  results 

Vc(z)  " n [^  - y^  * p(y  W ••  y)^J  • (16) 

No  formal  analysis  has  been  found  for  the  case  of  continuous  in- 
put, but  one  may  suppose  that  the  result  will  usually  resemble 
the  above  in  containing  terms  involving  respectively  ^V(y,a^)) 
(see  Appendix  III  for  notation)  and  V(a’),  and  further  that  the 
two  terms  will  have,  rather  than  a comnon  coefficient  l/n,  two 
distinct  coefficients  involving  different  effective  correlation 
times,  but  both  of  the  order  of 

Process  D 

Depending  on  whether  q''  or  a’  is  chosen  as  the  final  out- 
put, the  relevant  variance  is  V(q’)  or  V(a*).  These  are,  to  a 
sufficient  approximation,  related  by  a common  factor. 

^ - f2(a)  or  F2(a)  (17) 
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In  the  discrete-independent  input  case,  Y(q*)  is  essen- 
tially the  variance  of  a binomial  distribution.,^ 


V(q«)  ^ H „ 


(18) 


Again,  for  continuous  input,  formal  analysis  is  lackingo  One 
may,  however,  consider  that  n will  be  replaced  by  an  effective 
number  of  "independent  opportunities"  in  time  T for  the  alter- 
native events  \diose  probabilities  are  q and  po  This  number  may 
be  of  the  order  of  T/r^,  as  suggested  in  connection  with  process 
C. 


COMPARISON  OF  DETECTION  PROCESSES 

In  the  foregoing,  V^(z)  ^ Vb(z)  has  been  demonstrated 
for  the  cases  of  discrete- independent,  and  continuous,  input- 
Taken  in  conjunction  with  Eq-  (7),  this  indicates  that  process 
B generally  has  a lower  threshold  than  process  A-  No  general 
comparison  of  B and  C has  been  found  possible-  The  greater  sig- 
nal from  C is  offset,  surely  in  the  discrete-independent  case, 
and  probably  also  in  the  continuous,  to  some  degree  by  increased 
output  noise  arising  from  the  fluctuations  in  the  rejection  limit 
a* » In  view  of  the  analytical  difficulties  in  treating  process 
C,  actual  comparison  with  B in  specific  cases  will  probably  need 
to  be  done  empirically-  Similarly  with  process  D,  though  one 
may  suppose  here  that  the  greater  information  loss  involved  will 
usually  operate  against  the  effectiveness  of  the  process - 


(^NERALIZATION  OF  PROCESS  A 

One  may  generalize  the  rejection  process  by  replacing 
the  rejected  values  of  y with  a fixed  number  b,  rather  than  with 
zero-  This  then  includes  "clipping"  as  the  special  case  b "=  y[a]- 

Formally,  we  define  process  A*  as  like  process  A except 
that  y * b when  lx(t)i  > a-  One  finds,  for  output  signals 

Na*  * (yfxl  - b)  g(x)  dx 

" Nc  - (y  la)  - b)  [“g{x)]  dx-  (19) 
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This  result  includes  Eqs.  (2’),  (3’),  and  (5* ) as  special  cases. 

Turning  to  the  output  fluctuations,  we  will  make  only 
the  following  remarks.  Equation  (10)  is  replaced  by 

v^jCy)  “ q [iy^  - y^)  + p(t>  - y)^] 


and  Eq.  (13)  by 


VAt(z)  - (1/n)  Vy^,(y). 


(20) 


The  final  signal-to-noise  ratio  R for  the  discrete-independent 
input  case,  obtained  from  Eqs.  (1),  (19),  and  (20)  has  a maxi- 
mum with  respect  to  b,  at  fixed  a,  vdiich  is  readily  calculated 
and  occurs  for  b > y» 

The  modification  of  Eq.  (9)  for  this  process  is  straight- 
forward, but  will  not  be  given  here. 


LIKELIHOOD  SHAPING 


The  whole  of  this  section  rests  on  a suggestion  made  by 
T.  G.  Birdsall,  of  the  University  of  Michigan,  in  conversation 
with  the  writer.  We  observe  that  the  conqputations  of  Appendix  I 
determine  the  likelihood  ratio  for  an  instantaneous  value  of 
x(t),  say  L(x),  and  introduce  the  following  form  for  y; 


yW 


log  L(x)  - log  ^ 1 + fr*- 


(21) 


The  resulting  value  of  z is  an  exact  likelihood-detector 
output  if  the  input  x- values  are  discrete  and  independent,  and 
is  proposed  for  consideration  in  the  case  of  autocorrelated  in- 
put also.  Further  analysis  or  empirical  investigation  is  needed 
to  determine  the  extent  to  which  this  result  differs  from  the 
true  likelihood  output  for  an  autocorrelated  input. 

Sinplification  of  Eq.  (21)  is  permissible.  From  Eq.  (28), 
for  example,  we  have 
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The  appropriate  value  for  cr^m  this  expression  is  the  power  of 
a signal  which  the  process  is  designed  to  detect,  that  is  to  say, 
a threshold  signal.  Sett^g  from  the  first  line  of  Table  I, 
for  example,  equal  to  from  Eq.  (8)  shows  the  threshold 

cr^  to  be  of  the  order 

r 1/2 

[r  ’*<»'>  ] • 


Inserting  this  in  Eq.  (22)  gives 
/ r*  \^/2  . . 


L(x)  * 1 ♦ 


(^)  »<*) 


where 

w(x)  - 2f’(x) 

is  of  order  of  magnitude  unity  or  less.  Since  we  suppose  T>>JJ, 
L(x)  differs  little  from  unity  and  the  logarithm  in  Eq.  (21) 
can  be  expanded  in  a power  series,  of  vdiich  only  the  first  degree 
term  need  be  retained. 


y[xl  *(ip)  w(x) 


To  this  approximation,  constant  factors  have  no  effect  and  may 
be  omitted.  Thus  we  may  take,  finally 


d^f'>(x) 


f'(x) 
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or,  if  the  input  envelope  is  employed. 


yDd 


1 


(25) 


These  functions  maybe  called  likelihood  shapes  for  the  detector 
characteristic.  When  f*(x)  is  the  Gaussian  distribution,  Eqs. 
(24)  and  (25)  become,  essentially,  yW  ■ x^. 

From  Eq.(24)  one  may  readily  find  the  final  output  sig- 
nal-to-noise  ratio. 


is  the  correlation  time  for  the  time  series  y determined  by 
Eq.  (24).  Here  again  we  have  a factor  depending  only  on  the 
first-order  distribution,  and  the  factor  rj^  involving  also  the 
second  order  distribution  unless  the  input  values  are  discrete 
and  independent. 


SUMMARY  REMARKS 

The  foregoing  results  may  be  applied  to  study  small-signal 
detection  problems  in  which  the  background  statistics  are  not 
necessarily  Gaussian.  Empirically  determined  distribution  func- 
tions for  the  background  noise  may  be  used  to  calculate  or  esti- 
mate detection  thresholds.  Effects  of  the  following  character- 
istics of  the  detection  process  may  be  studied; 

a.  peak  rejection  or  clipping  with  various 
limits  (values  of  a). 

b.  various  detector  characteristics  (forms 
of  the  function  y[x]J  ). 

c.  the  special  two-valued  detector  char- 
acteristic (y  ■ 0 or  1)  implied  in  Pro- 
cess D. 
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do  the  special  detector  characteristics  Eq„ 

(24)  or  (25)  determined  by  likelihood 
shaping  o 

As  previously  pointed  out,  detection  thresholds  depend 
only  on  a first-order  distribution  if  the  input  values  are  dis- 
crete and  independent o In  the  more  usual  case  of  autocorrelated 
input,  most  of  the  processes  discussed  are  fully  determined  by 
the  second-order  distribution,  but  even  in  these  cases  it  is  to 
be  hoped  that  the  first-order  distribution  alone  may  be  roughly 
indicative o Processes  C and  D involve  statistics  beyond  the 
second  order. 
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APPENDIX  I.  EVALUATION  OF  g(x)  AND  G(x). 


Let  f®(x),  or  f*(x)  + g’(x)  be  the  probability  density 
for  X itself,  defined  over  - a?<x  < a?.  The  even  parts  of  these 
functions  willbe  f (x)  and  f (x)  + g(x) . The  density  f’(x)  g’(x) 
will  be  the  convolution  of  f*(x)  and  the  probability  density  for 
signal  alone,  s(x)<,.^ 


f»(x)  + g«(x)  ■=  /^f»(x  - u)  s(u)  du.  (27) 
-CO 

f’(x  - u)  is  now  expanded  as  three-term  power  series 
about  f’(x)  and  it  is  assumed  that  almost  all  values  of  u (in- 
stantaneous signal)  are  small  enough,  and  the  function  f’(x) 
smooth  enough  to  justify  the  use  of  the  expansion  in  Eq„  (27). 
Thus  we  have 


f’(x) 


+ 


ft(x)  - dfjlxj 

dx 


s(u)  du 


or 


g*(x)  " 1/2  (28) 

dx^ 


and 

g(x)  - 1/2  (29) 

dx^ 


recalling  that  the  mean  signal  is  supposed  zero. 

When  envelopes  of  noise  and  signal  are  considered,  the 
rule  of  combination  becomes  a vector  addition.  Let  the  noise 
only  and  signal  only  be  represented  by  N(t)  [coswt  + 0(t)]  and 
S(t)  j^cos  o/t  ♦ ^(t)J,  respectively.  The  envelopes  N(t)  and 
S(t),  and  the  phase  difference  0 “ 0(t)  -)?(t)  are  assumed  to  be 
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three  independent  random  variables,  with©  uniformly  distributed 
over  the  circle.  They  serve  to  determine  the  envelope  of  noise- 
plus-signal,  say  E,  by  the  relations 

2NS  cos©  + with  N,  S,  E all  ^ 0.  (30) 

Let  the  joint  probability  density  in  the  (N,  S,  ©)  space  be 
(l/2»)  F(N)  Sg(S),  which  then  serves  to  determine  the  distribu- 
tion of  E by 

[f(Ei)  + G(Ei)]  dE 

- (]/2t^  F(N)  Sg(S)  dN  dS  d©,  (31) 

The  region  of  integration  R is  the  volume  within  which 
Ej^  < E < Ej  dE. 

The  integral  in  Eq.  (31)  is  now  transformed  to  new  in- 
dependent variables  (E,u,  v)  given  by  u - S cos  ©,  v - S sin  ©, 
and  Eq.  (30).  The  Jacobian  is 


and  we  write  F(N)  ■ Fj^  (E,u,v).  Then 
[f(Ei)  + G(Ei)]  dE 

/*  2 *”1/2 

R h ^1  1 - ^ ) I dE  du  dv 

or  with  slight  simplification,  replacing  Ei  by  x 

- 4,  Fi(x,u,v)(l  - [2sl^]du  dx.  (33) 
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The  further  argument  is  restricted  to  x satisfying 


x2  >>  (34) 

that  is  to  X with  magnitude  greater  than  that  of  almost  all 
signal  values.  F]^  is  expanded  as  a power  series  in  u and  v,  in- 
cluding quadratic  terms,  about  (x,  0,  0).  After  some  reduction, 
one  obtains: 


F(x)  + G(x) 

./,  .Tni)  - „ . 1/2  1 Mil  y^l 

L dx  dx^  2x  dx  i 


It  is  now  to  be  noted  that  the  third  square  bracket  in the  inte- 
grand is  the  probability  density  in  the  (u,  v)  plane.  Because 
of  the  restriction  (34),  the  integration  is  over  almost  all 
values  of  u and  v.  Hence  the  effect  of  the  integration  is  merely 
to  replace  u,  u^,  and  v2  by  their  respective  mean  values  0, 
and  (taking  the  mean  of  to  be  2tt2),  have,  finally. 


G(x)  - 1/2  ^ F(x)]  <7-  (36) 

In  view  of  the  limitation  (34),  Eq.  (36)  is  not  correct,  for  a 
fixed  at  X ■ 0.  However,  without  further  analysis,  the  rela- 

tive error  involved  in  using  Eq.  (36)  over  the  range  0 ^ x ^ -a? 
will  be  assumed  small,  for  sm^l  o'  2,  The  limiting  behavior  of 
Eq.  (36)  as  X -►  0 seems  satisfactory. 

For  use  of  these  results  we  require  the  integrals: 

X,*  [-«(»)]  dl  . - 1/2  ^ .-2  (37) 


25 


SIO  Reference  58-34 


since 
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APPENDIX  II.  EFFECT  OF  CLOSING  GAPS 


Equation  (12)  gives  the  output  variance  V(2)  for  a con- 
tinuous input  containing  gaps,  as  in  the  upper  part  of  Figure  2. 
To  show  the  effect  of  closing  the  gaps,  we  separate  the  proba- 
bility density  £2  into  components  as  follows; 


f2(u,v,r) 


(39) 


^^o  is  probability  density  for  ordinate  pairs  (u,v) 

separated  by  an  interval  r containing  no  rejection  gaps; 
fl  (u,v,T,  w)  dw  is  the  probability  density  for  such  pairs  when 
the  interval  r contains  rejection  gaps  of  total  duration  lying 
between  w and  w + dw.  Equations  (8)  and  (9)  may  then  be  written 

” T dr[-f(u)  f(v)  + fQ(u,v,r) 

♦ /q  fi(u,v,r,  w)jdw  . (40) 

For  large  r , x(t)  and  x(t  + r)  are  presumed  to  vary  independ- 
ently. The  square  bracket  in  the  integrand  of  Eq.  (40)  then 
vanishes  for  large  r;  the  integration  with  respect  to  r is  assumed 
to  converge  and  the  upper  limit  may  be  set  at  a sufficiently 
large  positive  number  L rather  than  co  . 

The  double  integral  in  Eq.  (40)  with  respect  to  w and  r 
will  now  be  transformed  to  new  independent  variables  w and 
f®r-  w.  (An  ordinate  pair  originally  characterized  by  (r,  w) 
will  have  the  separation  r’  after  gaps  are  closed.)  The  Jacobian 
is  unity.  The  original  region  of  integration  is  shown  in  Figure 
4,  together  with  contours  without  which  the  density  f^  effective- 
ly vanishes.  These  represent  the  assumed  behavior, that  as  r-^oo, 
the  rejected  fraction  within  the  interval  r becones  increasingly 
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Figure  4.  Regions  of  integration.  Eq.{40)  'A 
Eg.  {4i)  • DCOE.  The  probability  det 

fg  vanishes  apprcximatefy  above  the 
contour  c and  below  c'. 
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predictable  and  w/r-»  (1  -q)  is  the  only  region  of  nonvanishing 
fj.  Hence  the  region  of  integration  may  be  altered  to  0<x<<», 
0 <r»  <qL  as  also  shown  in  Figure  4.  For  the  term  involving  the 
constant  f(u)  f(v)  we  write  dr*  dr’/q,  but  because  fgCu^VjL) 
vanishes  approximately,  we  assume 

Jo  fo(u,v,r»)dr*. 

Hence  the  final  transformed  expression  can  be  written 
(using  r for  r* )s 


V^(z)  ^ ^ J du  dv  y[u]  y[v]  y®  dr 

. ♦|i  |fo(u,v,r)  ♦ f]^(u,v,r  ♦ w,w)  dw|j| 


- Vb(2)  (41) 

The  relation  to  V3  follows  when  one  observes  that  the 
term  in  the  integrand  which  is  in  square  brackets  is  the  proba- 
bility density  analogous  to  Eq.  (39)  into  which  £2  must  be 
changed  by  the  gap-closing  operation. 
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APPENDIX  III.  CALCULATION  OF  Vc(z). 


The  case  of  discrete,  independent  input  values  is  con- 
sidered. Of  the  n values  of  x occurring  during  T,  the  lowest 
nq  are  retained  and  will  be  referred  to  as  an  x-sample.  These 
samples  are  classified  into  subgroups  according  to  the  value  of 
a*.  Within  one  subgroup  the  x-values  are  distributed  independ- 
ently over  the  interval  0 <x  <a’,  yielding  the  following  mean 
and  variance  for  y; 


E{y,a») 


V(y,a») 


f(x)  dx 

/o*’f(x)  dx 


/o^’y^W  fU)  dx 


f(x)  dx 


E^(y,a*). 


We  use  < > to  denote  a mean  with  respect  to  a* , and 
note  that  in  this  case  z can  be  regarded  a^  a mean  of  (nq)  values 
of  (qy),  in  which  y means  a retained  value.  Hence 

Eq(z)  - q < E(y,a’)> 

Vj,(z)  -<V(z,a»)>  ♦ <E2(z,a»)>  - <E(z,a«)>^ 

" [^<V(y,a»)>  + <E2(y,a»)>  - <E(y,a*)>^]  • 


We  take,  approximately s 
<V(y,a»)>"  V(y,a)  - ^ - y^ 
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2 

<E2(y,  a»)>-<E(y,  a*)>^  “ ( 

dE^y.a)  - iM  (y Qi]  > y) 

da*  q 

and 


V(a*) 


- M 

nf2(a) 


using  Eqs,  (17)  and  (18).  Coinbination  of  the  last  five  equa- 
tions leads  to  Eq.  (16). 


cotnou  iWNEisin 

**ww  uamiowfs 
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Cambridge  Research  Center 
Air  Research  & Development 
Conmand 

L,  G.  Hanscom  Field 
Bedford,  Massachusetts 
Copy  18 

Conmanding  Officer  & Director 
David  Taylor  Model  Basin 
Washington  7,  Do  Co 
Copy  19 

Comnanding  Officer  b Director 
Naval  Ordnance  Laboratory 
White  Oak 

Silver  Spring  19,  Maryland 
Copy  20 

Commanding  Officer  b Director 
Naval  Research  Laboratory 
Washington  25,  Do  Co 
Codes  2021  and  5500 
Copy  21 

Commanding  Officer  b Director 
Uo  So  Navy  Electronics  Laboratory 
San  Diego  52,  California 
Copies  22  b 23 

Conmanding  Officer  & Director 
U.SoNo  Engineering  Experiment 
Station 

Annapolis,  Maryland 
Copy  24 
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Commanding  Officer  & Director 
U.  S.  Navy  Mine  Defense 
Laboratory 

Panama  City,  Florida 
Copy  25 

Connanding  Officer  & Director 
U.S.N.  Underwater  Sound 
Laboratory 
Fort  Truniull 
New  London,  Connecticut 
Copy  26 

Director 

U.S.N.  Underwater  Sound 
Reference  Laboratory 
P,  0,  Box  3629 
Orlando,  Florida 
Copy  27 

Superintendent 
Naval  Postgraduate  School 
Monterey,  California 
Attns  Librarian 
Copy  28 

Bureau  of  Ships 
Department  of  the  Navy 
Washington  25,  D.  C. 

Code  209 

Copies  29,  30,  31,  32, 

33,  34,  35,  36, 

37  & 38 

President 

U.  S.  Army  Beach  Erosion  Board 
5201  Little  Falls  Road,  NW 
Washington  16,  D.  C. 

Via;  Chief  of  Array 
Engineers 
Copy  39 

Executive  Secretary 
Comnittee  on  Undersea  Warfare 
National  Research  Council 
2101  Constitution  Avenue,  NW 
Washington  25,  D.  C. 

Via;  Contract  Administrator 
Southeastern  Area 
2110  G.  Street,  NW 
Washington  7,  D.  C. 

Copy  40 


Bell  Telephone  Laboratories,  Inc. 
Whippany,  New  Jersey 

Via;  Inspector  Naval  Material 
Naval  Industrial 
Reserve  Shipyard 
Building  13,  Port  Newark 
Newark  5,  New  Jersey 
Copy  41 

Hughes  Tool  Company 
Aircraft  Division 
Culver  City,  California 

Via;  Bureau  of  Aeronautics  Rep. 
Inglewood 

North  American  Aviation,  Inc 
Los  Angeles  45,  California 
Copy  42 

Director 

Woods  Hole  Oceanographic 
Institution 

Woods  Hole,  Massachusetts 

Via;  ONR,  Boston,  Massachusetts 
Copy  43 

Brown  University 
Research  Analysis  Group 
108  Hope  Street 
Providence  6,  Rhode  Island 

Via;  ONR,  Boston,  Massachusetts 
Copy  44 

Columbia  University 
Director 

Lamont  Geological  Observatory 
Palisades,  New  York 

Via;  ONR,  New  York  13,  New  York 
Copy  45 

Columbia  University 
Director 

Hudson  Laboratories 
P.  0.  Box  239 
Dobbs  Ferry,  New  York 

Via;  ONR,  New  York  13,  New  York 
Copy  46 

Dr.  F,  V.  Hunt 
c/o  Office  of  Naval  Research 
Branch  Office 
Boston,  Massachusetts 
Copy  47 
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Chesapeake  Bay  Institute 
The  Johns  Hopkins  University 
Bailtimore  18 5 Maryland 
Vias  Naval  Research 
Resident  Rep. 

The  Johns  Hopkins  Univo 
Whitehead  Hall 
34tri  & Charles  Street 
Baltimore,  Maryland 
Copy  48 

Yale  University 
Director 

Laboratory  of  Marine  Physics 
Box  1916,  Yale  Station 
New  Haven,  Connecticut 

Vias  ONR,  New  York  13,  N,Y, 
Copy  49 

Pennsylvania  State  University 
Ordnance  Research  Laboratory 
Po  0«  Box  30 

University  Park,  Pennsylvania 
Vias  Devo  Contract  Repo  ORL 
Copy  50 

University  of  Michigan 
Electronics  Defense  Group 
Ann  Arbor,  Michigan 
Attns  To  Go  Birdsall 
Vias  Central  Industrial 

Security  Regional  Office 
ARDC,  Wright“Patterson 
Airforce  Base,  Ohio 
Copy  51 

University  of  Texas 
Director 

Defense  Research  Laboratory 
P,  0,  Box  8029 
Austin,  Texas 

Vias  ONR,  ReSo  Repo 
Copy  52 


University  of  Washington 
Director 

Applied  Physics  Laboratory 
Seattle,  Washington 

Vias  Inspector  Naval  Material 
2300  Eleventh  Avenue,  SW 
Seattle,  Washington 
Copy  53 

University  of  Washington 
Director 

Oceanographic  Laboratories 
Seattle,  Washington 

Vias  Inspector  Naval  Material 
2300  Eleventh  Avenue,  SW 
Seattle,  Washington 
Copy  54 

University  of  California 
Graduate  Division 
405  Hilgard  Avenue 
Los  Angeles  24,  California 

Vias  ONR,  Pasadena,  California 
Copy  55 

University  of  California 
Scripps  Institution  of  Oceanography 
La  Jolla,  California 
Director,  Copy  56 
Library,  Copy  57  b 58 
Geophysics,  Copy  59 
Oceanographic  Publications,  Copy  60 
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